Skip to content

ladybug: Add versions 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.20.4, 0.21.0, 0.21.1, 0.21.2 - #2475

Draft
riseproject-dev[bot] wants to merge 3 commits into
mainfrom
github-actions/nightly-upgrade/ladybug
Draft

riseproject-dev[bot] wants to merge 3 commits into
mainfrom
github-actions/nightly-upgrade/ladybug

Conversation

@riseproject-dev

@riseproject-dev riseproject-dev Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Automatically generated by the nightly check_versions.py run.

ladybug v0.19.1 -> v0.20.0, v0.20.1, v0.20.2, v0.20.3, v0.20.4, v0.21.0, v0.21.1, v0.21.2

Every - version: entry added to docs/packages/ladybug.yaml is built by this PR's own build-ladybug.yml run; merging publishes the wheels.

@riseproject-dev
riseproject-dev Bot requested a review from luhenry September 29, 2026 08:01
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://riseproject-dev.github.io/python-wheels/pr-preview/pr-2475/

Built to branch gh-pages at 2026-10-02 21:48 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@riseproject-dev
riseproject-dev Bot force-pushed the github-actions/nightly-upgrade/ladybug branch from 76f5e4c to 447efd9 Compare September 30, 2026 08:09
@riseproject-dev riseproject-dev Bot changed the title ladybug: Add versions 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.20.4, 0.21.0 ladybug: Add versions 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.20.4, 0.21.0, 0.21.1 Sep 30, 2026
@luhenry
luhenry force-pushed the main branch 3 times, most recently from 39fb7ba to a75cf68 Compare October 1, 2026 15:22

luhenry commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Build ladybug 0.20.0 cp314t-manylinux_riscv64 segfaulted (exit 139) partway through the test suite, in test_blob_parameter.py::test_bytes_param — a plain blob/bytes round-trip test with no threading of its own, so this doesn't look like a deterministic test-design gap (unlike test_compile_release_on_another_thread-style free-threading test bugs elsewhere in this campaign). It reads more like undefined-behavior-driven non-determinism from running this pybind11-based native extension under a free-threaded interpreter it was never built against (LBUG_PYTHON_BACKEND=pybind; pybind11 has no official free-threading support without an explicit opt-in the vendored bindings don't set) — consistent with this workflow already carrying several other cp314t-only test-file ignores from earlier rounds.

I can't rerun the failed job yet (rerun-failed-jobs refuses while the parent run is still active — 27 other build jobs across the other 6 versions are still queued). Holding off on a bigger structural fix (e.g. dropping cp314t from the matrix entirely) until I see whether this reproduces on the other versions' cp314t legs too, which would confirm it's systemic rather than a one-off memory-safety fluke. Will follow up once more of the matrix has run.

luhenry commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Update: two more distinct failures have come in from this version/interpreter matrix (7 versions × 4 interpreters = 28 jobs, still mostly in flight):

  • 0.20.3 cp313-manylinux_riscv64: "Build failed because docker is too old or is not working properly" — the known self-hosted-runner Docker-daemon infra flake, unrelated to the segfault above. Will include in the batch rerun once the whole run finishes (can't rerun individual jobs while the parent run is still active).
  • 0.20.2 cp314-manylinux_riscv64 (not t — a regular GIL build): FAILED tools/python_api/test/test_scan_pandas_pyarrow.py::test_pyarrow_primitive - Failed: tables are not equal. This is a clean, deterministic assertion failure, not a crash, and it's on a non-free-threaded interpreter — so it doesn't fit the earlier segfault's "free-threading + pybind11" theory at all. Looks like a separate, real regression (likely a pandas/pyarrow version-resolution difference pulled in at build time - the same test file logs several DeprecationWarnings about NumPy timedelta units going away, suggesting the installed pandas/pyarrow versions shifted under this version of ladybug). Haven't dug into root cause yet.

Holding off on any structural fix until more of the matrix finishes - with three different failure shapes across three different (version, interpreter) pairs so far, I want to see the full picture (which versions/interpreters are actually clean) before deciding what, if anything, needs a real fix versus just a rerun.

luhenry commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Root-caused the segfault and the concurrent-query failure — both are 0.20.0-only, confirmed-fixed-upstream bugs, not riscv64-specific:

  • test_bytes_param (test_blob_parameter.py) re-executes the same parameterized CREATE string through the implicit prepared-statement cache. On 0.20.0 that SIGSEGVs on the second execution: FactorizedTable::clear() dereferences a null block collection for a write statement's empty-schema result. Fixed upstream in ladybugdb/ladybug@d09008e ("Fix SIGSEGV re-executing a parameterized write query string (opencv-contrib-python riscv64 support #862)"), not released until 0.20.1.
  • test_async_prepare_and_execute_concurrent re-executes a parameterized read ($1) concurrently through the same cached-plan fast path, and fails the same way on 0.20.0 only. Most likely ladybugdb/ladybug@47443cd ("Fix two state-reuse bugs on the cached-physical-plan fast path"), landing in the same 0.20.0..0.20.1 range.

Confirmed version-scoped, not GIL-related: both failed identically on 0.20.0 cp314 and cp314t (same stack frame, same line), while 0.20.2 cp314t and 0.20.3 cp314t built and tested clean — ruling out the free-threading theory from my earlier comments. Pushed a fix that deselects both tests for matrix.version == '0.20.0' only, rather than masking them across every version.

Two other failures remain open, unrelated to the above:

  • 0.20.3 cp313: known Docker-daemon infra flake (unrelated to this diff) — will batch a rerun once the rest of the 28-job matrix clears so the API stops 403'ing on "already running".
  • 0.20.2 cp314: test_pyarrow_primitive ("tables are not equal") — this test generates its fixture data with unseeded random.randrange()/getrandbits() every run, so it's plausibly this run's own data draw rather than a reproducible bug. Will retry the same way once the matrix clears; if it recurs with different random data it's a real bug worth root-causing further.

luhenry commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Update: the per-test deselect from the previous comment was incomplete. A new matrix run surfaced a third crash — test_datatype.py::test_large_array — hitting the identical connection.py:733 _execute_with_pybind segfault. It has the same shape as test_bytes_param: a loop re-executing one parameterized CREATE query on the same connection. A static grep of the whole suite for that pattern turned up a fourth, test_udf.py's udf_predicate_test helper (used by test_udf).

Rather than keep whack-a-moling individual tests as the matrix finds them, backported the two actual upstream C++ fixes instead:

  • ladybugdb/ladybug@d09008e → patches/ladybug/0.20.0/0004-...patch (the FactorizedTable::clear() null-deref on empty-schema tables)
  • ladybugdb/ladybug@47443cd → patches/ladybug/0.20.0/0005-...patch (the ResultCollector::prepareForReuse() live-QueryResult clobber — upstream's own commit message cites test_async_prepare_and_execute_concurrent by name: "asserted [96] == [1]", confirming that test's root cause too)

Both verified to apply cleanly against the real v0.20.0 tree (not just the submodule) and pass ci_scripts/check_patch.py. Dropped the now-unnecessary matrix.version == '0.20.0' deselect from CIBW_TEST_COMMAND since the real fix covers all four (and anything else sharing the pattern) at the source instead.

… 0.21.1, 0.21.2

Signed-off-by: riseproject-dev[bot] <330740410+riseproject-dev[bot]@users.noreply.github.com>
@riseproject-dev
riseproject-dev Bot force-pushed the github-actions/nightly-upgrade/ladybug branch from 9493518 to 4a00153 Compare October 2, 2026 08:03
@riseproject-dev riseproject-dev Bot changed the title ladybug: Add versions 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.20.4, 0.21.0, 0.21.1 ladybug: Add versions 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.20.4, 0.21.0, 0.21.1, 0.21.2 Oct 2, 2026
…he bot's version bump dropped

The nightly-upgrade bot force-pushes this branch from main plus its own
docs/packages/ladybug.yaml bump every time check_versions.py finds a
newer upstream release while the PR is still open, which silently
discards every commit this branch carried that the bot doesn't know
about - in this case all of it: the base riscv64-enablement patches
for 0.20.0-0.21.1 (VMRegion reservation size, the interrupt-repeat
test fix, and the riscv64 platform-extension report), 0.20.0's
SIGSEGV backport, and 0.20.2's empty-pyarrow-scan backport. Restoring
all three, plus carrying the same base patch set forward into the
newly added 0.21.2 since it needs it for the same reason 0.20.0-0.21.1
originally did.
test_mvcc_bank.py::test_multi_writer_no_anomalies aborted once, on the
0.20.3 cp314t leg. glibc's assertion fired in pthread_mutex_lock under
TaskScheduler::pushTaskIntoQueue. taskSchedulerMtx is a plain
std::mutex, and every access to it goes through that lock, so a mutex
whose owner field is already set means something else overwrote its
memory. This is not a memory-ordering bug in the scheduler, and none of
this branch's patches touch that code: the VMRegion fallback never runs
with the test's 1 GiB max_db_size.

Free threading is not the cause either. PyConnection::query releases
the GIL around Connection::query on every interpreter, and _lbug's
PYBIND11_MODULE declares no Py_mod_gil, so cp314t turns the GIL back
on at import. All four legs run the same concurrent C++. The same
upstream test file passed on the 0.20.2, 0.20.4, 0.21.0 and 0.21.1
cp314t legs, where 0.20.4 has the same python_api commit as 0.20.3,
and on every GIL leg. 32 runs of upstream's x86_64 cp314 0.20.3 wheel
also passed. That fits a rare race in enable_multi_writes. Upstream's
CI tests only CPython 3.12 on x86_64 and ships no cp314t wheel.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant